Using and Evaluating Differential Modeling in Intelligent Tutoring and Apprentice Learning Systems

A powerful approach to debugging and refining the knowledge structures of a problem-solving agent is to differentially model the actions of the agent against a gold standard. This paper proposes a framework for exploring the inherent limitations of such an approach when a problem solver is differentially modeled against an expert system. A procedure is described for determining a performance upper bound for debugging via differential modeling, called the synthetic agent method. The synthetic agent method systematically explores the space of near miss training instances and expresses the limits of debugging in terms of the knowledge representation and control language constructs of the expert system.